Use of Internal Models to Measure Market Risks for Investment Guarantees under MPF Schemes

I@ HONG KONG MONETARY AUTHORITY

• {jfm~IIl~~r.u
  CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes This module should be read in conjunction with the Introduction and with the Glossary, which contains an explanation of abbreviations and other terms used in this Manual. If reading on-line, click on blue underlined headings to activate hyperlinks to the relevant module. Purpose To prescribe standards and criteria for adopting a model-based approach to the measurement of market risks for investment guarantees under MPF schemes Classification A technical note issued by the MA Previous guidelines superseded This is a new guideline. Application To all Als which use internal models to measure market risks for investment guarantees under MPF schemes Structure

1. Introduction
   1.1 Terminology
   1.2 Qualitative standards
2. Specification of market risk factors
   2.1 General
   2.2 For interest rates
   2.3 For exchange rates (including gold)
   2.4 For equity prices
   2.5 For commodity prices
3. Quantitative standards

HONG KONG MONETARY AUTHORITY
~. 1f~16:ti'lfl!l!rni
CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
3.1 General
3.2 Minimum standards
3.3 Daily capital requirement
3.4 Multiplication factor
3.5 Specific risk capital charge
4. Treatment of specific risk
4.1 General
4.2 Criteria
4.3 Surcharge
4.4 Back-testing requirement
4.5 Exception analysis
5. Model review
5.1 Acceptability criteria
5.2 Portfolio testing
5.3 Recognition of models
6. The use of back-testing
6.1 Introduction
6.2 Description of the back-testing framework
7. The interpretation of back-testing results
7.1 Three zone approach
7.2 Green zone
7.3 Yellow zone
7.4 Red zone
7.5 Plus factor for back-testing exceptions
8. Classification of exceptions
8.1 Reasons for exceptions
8.2 Discussion of categories
8.3 Consideration of factors by HKMA

HONG KONG MONETARY AUTHORITY
~. ~m~M!lL~J!I!fflj
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes

1. Introduction
   1.1 Terminology
   1,1.1 See CA-S-4 "Capital Adequacy Requirements for Investment Guarantees under MPF Schemes" for an explanation of common terms used in the modules relating to MPF guaranteed funds.
   1.2 Qualitative standards
   1.2.1 Als that use a model-based approach to measure market
   risks for investment guarantees under MPF schemes must comply with certain qualitative criteria. The extent to which they meet these may influence the level at which the HKMA will set the multiplication factor in subsection 3.4 below. Only Als whose models comply fully will be eligible for application of the minimum multiplication factor.
   1.2.1 The qualitative criteria include:

• an independent risk control unit to review market
  risk measured by the models;
• the complete scope of market risks being
  captured by the risk measurement model;
• accurate and complete position data;
• timely, reliable and independent data sources
  being used to run internal models;
• accurate and appropriate volatility and correlation
  assumptions;
• accurate valuation and risk transformation
  calculations; and
• verification of the model's accuracy through
  frequent back-testing as described in subsection
  6.2 of this module.

HONG KONG MONETARY AUTHORITY

• :m:~~I!l~$rnJ
  CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes

2. Specification of market risk factors
   2.1 General
   2.1.1 The risk factors contained in a market risk measurement
   system should be sufficient to capture the risks inherent in the guaranteed investment fund's portfolio of on- and off-balance sheet positions. Although Als will have some discretion in specifying the risk factors for their internal models, the following should be met.
   2.2 For interest rates
   2.2.1 There must be a set of risk factors corresponding to
   interest rates in each currency in which the investment fund has interest rate sensitive on- or off-balance sheet positions.
   2.2.2 The risk measurement system should model the yield
   curve using one of a number of generally accepted approaches, e.g. by estimating the forward rates of zero coupon yields. The yield curve should be divided into various maturity segments in order to capture variations in the volatility of rates along the yield curve. There will typically be one risk factor corresponding to each maturity segment.
   2.2.3 For material exposures to interest rate movements in the
   major currencies and markets, Als must model the yield curve using a minimum of six risk factors. The number of risk factors used, however, should ultimately be driven by the nature of the investment fund's strategies. For instance, a portfolio of various types of securities across many points of the yield curve and that engages in complex arbitrage strategies would require a greater number of risk factors to capture interest rate risk accurately.
   2.2.4 The risk measurement system must incorporate separate
   risk factors to capture spread risk (e.g. between bonds and swaps). A variety of approaches may be used to capture the spread risk arising from less than perfectly correlated movements between government and other fixed-income interest rates, such as specifying a completely separate yield curve for non-government

HONG KONG MONETARY AUTHORITY \i" ::m=m~Ill~J!I!fiU CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes fixed-income instruments (e.g. swaps or municipal securities) or estimating the spread over government rates at various points along the yield curve.
2.3 For exchange rates (including gold)
2.3.1 The risk measurement system should incorporate risk
factors corresponding to the individual foreign currencies in which the investments of the fund are denominated. Since the value-at-risk figure calculated by the risk measurement system will be expressed in Hong Kong dollars, any net position denominated in a foreign currency will introduce a foreign exchange risk. There must thus be risk factors corresponding to the exchange rate between Hong Kong dollars and each foreign currency in the investment fund.
2.4 For equity prices
2.4.1 There should be risk factors corresponding to each of the
equity markets in which the investment fund holds significant positions.
2.4.2 At a minimum, there should be a risk factor that is
designed to capture market wide movements in equity prices (e.g. a market index). Positions in individual securities or in sector indices can be expressed in "beta equivalents'" relative to this market wide index.
2.4.3 A more detailed approach would be to have risk factors
corresponding to various sectors of the overall equity market (for instance, industry sectors or cyclical and non￾cyclical sectors). As above, positions in individual stocks within each sector can be expressed in beta equivalents relative to the sector index.
2.4.4 The most thorough approach would be to have risk
factors corresponding to the volatility of individual equity issues. J A "beta equivalent" position would be calculated from a market model of equity price returns (such as the CAPM model) by regressing the return on the individual stock or sector index on the risk-free rate of return and the return on the market index.

'~ HONG KONG MONETARY AUTHORITY 1\. =m=~~IIt1fl!!r.u CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
2.4.5 The sophistication and nature of the modelling technique
for a given market should correspond to the investment fund's exposure to the overall market as well as its concentration in individual equity issues in that market.
2.5 For commodity prices
2.5.1 There should be risk factors corresponding to each of the
commodity markets in which the investment fund holds significant positions.
2.5.2 For relatively limited positions in commodity-based
instruments, a straightforward specification of risk factors would be acceptable. Such a specification would probably entail one risk factor for each commodity price. In cases where the aggregate positions are quite small, it might be acceptable to use a single risk factor for a relatively broad sub-category of commodities (e.g. a single risk factor for all types of oil).
2.5.3 For more active trading the model should encompass:

• directional risk to capture the exposure from
  changes in spot prices arising from net open positions;
• forward gap and interest rate risk to capture the
  exposure to changes in forward prices arising from maturity mismatches; and
• basis risk to capture the exposure to changes in
  the price relationships between two similar but not identical commodities.
  2.5.4 For more active trading, the model must also take
  account of variation in the "convenience yield"2 between derivative positions such as forwards and swaps and cash positions in the commodity. 2 The convenience yield reflects the benefits from direct ownership of the physical commodity (for example, the ability to profit from temporary market shortages) and is affected both by market conditions and by factors such as physical costs.

I@ HONG KONG MONETARY AUTHORITY

• ::m:m~IIl1f~rnJ
  CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes

3. Quantitative standards
   3.1 General
   3.1,1 Als have flexibility in devising the precise nature of their models but the following minimum standards will apply for the purpose of calculating their capital charges using the value-at-risk approach.
   3.2 Minimum standards
   3.2.1 Value-at-risk must be computed on a daily basis.
   3.2.2 In calculating the value-at-risk, a 99th percentile, one￾tailed confidence interval is to be used.
   3.2.3 In calculating value-at-risk, an instantaneous price shock
   equivalent to a 20-day movement in prices is to be used, i.e. the minimum holding period will be twenty trading days. Als may use value-at-risk numbers calculated according to shorter holding periods scaled up to twenty days by the square root of time (for the treatment of options, see also para. 3.2.8 below).
   3.2.4 The choice of historical observation period (sample
   period) for calculating value-at-risk will be constrained to a minimum length of one year. For Als that use a weighting scheme or other methods for the historical observation period, the effective observation period must be at least one year, i.e. the weighted average time lag of the individual observations cannot be less than six months. The HKMA may also require an AI to calculate its value-at-risk using a shorter observation period if, in the judgement of the HKMA, this is justified by a significant upsurge in price volatility.
   3.2.5 Als should update their data sets at least once every
   three months and should also reassess them whenever market prices underlying the assets of the investment fund are subject to material changes.
   3.2.6 No particular type of model is prescribed. Als will be free
   to use models based, for example, on variance￾covariance matrices, historical simulations or Monte Carlo simulations.

I .. ifm:&:IIl~:@Jjjij
CA-S-5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
3.2.7 Als will have discretion to recognise empirical
correlations within broad risk categories, e.g. interest rates, exchange rates, equity prices and commodity prices, including related options volatilities in each risk factor category. Empirical correlations across broad risk factor categories are also allowed, provided that the HKMA is satisfied that the AI's system for measuring correlations is sound and implemented with integrity.
3.2.8 Als' models must capture accurately the unique risks
associated with options within each of the broad risk categories. The following criteria apply to the measurement of options risk:

• Als' models must capture the non-linear price
  characteristics of options positions, e.g. volatility risk and gamma risk;
• Als are expected to move ultimately towards the
  application of a full 20-day price shock to options positions or positions that display option like characteristics. In the interim, the HKMA may require Als to adjust their capital measure for options risk through other methods, e.g. periodic simulations or stress testing; and
• each AI's risk measurement system must have a
  set of risk factors that captures the volatilities of the rates and prices underlying option positions, Le. vega risk. An investment fund with relatively large or complex options portfolios should have detailed specifications of the relevant volatilities. This means that Als should measure the volatilities of options positions broken down by different maturities.
  3.3 Daily capital requirement
  3.3.1 Each AI must meet, on a daily basis, a capital
  requirement expressed as the higher of:
• an average of the daily value-at-risk measures on
  each of the preceding sixty business days, multiplied by a multiplication factor; and

'" w~~lMI1fl!l!fiU
CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes

• its previous day's value-at-risk number measured
  according to the parameters specified in this
  section.
  3.4 Multiplication factor
  3.4.1 The multiplication factor will be set by the HKMA on the
  basis of its assessment of the quality of the AI's use of models (see subsection 1.2 above), subject to an absolute minimum of three. In addition, Als will be required to add to this factor a "plus factor" directly related to the length of notice period and the ex-post performance of the model. The notice period is the time period that a guarantor AI has to notify members of the investment fund in advance when the guarantor intends to modify or cancel the guarantee. The plus factor can be added when the notice period is longer than six months. Section 6 presents in detail the approach to be applied for back-testing and the plus factor.
  3.5 Specific risk capital charge
  3.5.1 Als using models will also be SUbject to a separate
  capital charge to cover the specific risk (as defined under the standardised approach) of interest rate related instruments and equity securities. The manner in which the specific risk capital charge is to be calculated is set out in section 4 below.

4. Treatment of specific risk
   4.1 General
   4.1.1 Als using models will be permitted to base their specific
   risk capital charge on modelled estimates if they meet all of the quantitative requirements for general risk models as well as the additional criteria set out below.
   4.1.2 Als which are unable to meet these additional criteria will
   be required to base their specific risk capital charge on the full amount of the standardised approach based specific risk charge.

1% '" :m:~:&IIl'iflM.f.ij
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
4.2 Criteria
4.2.1 The criteria for applying modelled estimates of specific
risk require that an AI's model:

• explains the historical price variation in the
  portfolio3
  ;
• demonstrably captures concentration (magnitude
  and changes in composition)4;
• be robust to an adverse environment5
  ; and
• be validated through back-testing aimed at
  assessing whether specific risk is being captured accurately.
  4.2.3 In addition, the AI must be able to demonstrate that it
  has methodologies in place which allow it to capture event and default risk adequately for its traded debt and equity positions.
  4.3 Surcharge
  4.3.1 Als which meet the criteria set out above for models but
  do not have methodologies in place to capture event and default risk adequately will be required to calculate their specific risk capital charge based on the internal model 3 The key ex-ante measures of model quality are "goodness-of-fit" measures which address the question of how much of the historical variation in price value is explained by the model. One measure of this type which can often be used is an R-squared measure from regression methodology. If this measure is to be used, the AI's model would be expected to be able to explain a high percentage. such as 90%, of the historical price variation or to include explicitly estimates of the residual variability not captured in the factors included in this regression. For some types of models, it may not be feasible to calculate a goodness￾of-fit measure. In such cases the AI is expected to work with the HKMA to define an acceptable alternative measure which would meet this regulatory objective. 4 The AI would be expected to demonstrate that the model is sensitive to changes in portfolio construction and that higher capital charges are attracted for portfolios that have increasing concentrations. S The AI should be able to demonstrate that the model will signal rising risk in an adverse environment. This could be achieved by incorporating in the historical estimation period of the model at least one full credit cycle and ensuring that the model would not have been inaccurate in the downward portion of the cycle. Another approach for demonstrating this is through simulation of historical or plausible worst-case environments.

HONG KONG MONETARY AUTHORITY
., 1!fm~lIl~l!I!}ffl
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes measurements plus an additional prudential surcharge as defined in the following paragraph.
4.3.2 For Als applying the surcharge, the total market risk
capital requirement will equal a minimum of three times the internal model's general and specific risk measure plus a surcharge of either:

• the specific risk portion of the value-at-risk
  measure which should be isolated according to supervisory guidelines6 ; or, at the AI's option
• the value-at-risk measures of sub-portfolios of
  debt and equity positions that contain specific risk? 6 Techniques for separating general market risk and specific risk would include the following:
  Equities
• The market should be identified with a single factor that is representative of the market
  as a whole, for example, a widely accepted, broadly-based stock index for the country concerned.
• Als that use factor models may assign one factor of their model, or a single linear
  combination of factors, as their general market risk factor. BQm1s
• The market should be identified with a reference curve for the currency concerned. For
  example, the curve might be a government bond yield curve or a swap curve; in any case, the curve should be based on a well-established and liquid underlying market and should be accepted by the market as a reference curve for the currency concerned.
• Als may select their own technique for identifying the specific risk component of the
  value-at-risk measure for purposes of applying the multiplier of 4. Techniques would include:
  (- using the incremental increase in value-at-risk arising from the modelling of specific risk factors; (- using the difference between the value-at-risk measure and a measure calculated by substituting each individual equity position by a representative index; or (- using an analytic separation between general market risk and specific risk implied by a particular model. 7 This would apply to sub-portfolios containing positions that would be subject to specific risk under the standardised approach.

HONG KONG MONETARY AUTHORITY
~. lfm~JiIl~J!!!r.u
CA·S·5 Use of Internal Models to Measure V. 1 ·16.02.01 Market Risks for Investment Guarantees under MPF Schemes
4.3.3 Als using the second option are required to identify their
sub-portfolio structure ahead of time and should not change it without the HKMA's consent.
4.3.4 The surcharge is designed to treat the modelling of
specific risk on the same basis as a general market risk model that has proven deficient during back-testing. That is, the equivalent of a multiplication factor of four would apply to the estimate of specific risk until such time as an AI can demonstrate that the methodologies it uses capture event and default risk adequately. Once an AI is able to demonstrate this, the minimum multiplication factor of three can be applied. A higher multiplication factor of four on the modelling of specific risk would remain possible, however, if future back-testing results were to indicate a serious deficiency with the model.
4.4 Back-testing requirement
4.4.1 Als which apply modelled estimates of specific risk are
required to conduct back-testing aimed at assessing whether specific risk is being captured accurately.
4.4.2 The methodology an AI should use for validating its
specific risk estimates is to perform separate back-tests on the investment fund's sub-portfolios using daily data on sub-portfolios subject to specific risk. The key sub￾portfolios for this purpose are traded debt and equity positions. If, however, an AI itself separates the investment fund's portfolio into finer categories (e.g. emerging markets, traded corporate debt), it is appropriate to keep these distinctions for sub-portfolio back-testing purposes.
4.4.3 Als are required to commit to a sub-portfolio structure
and to use it consistently unless it demonstrates to the HKMA that there is good reason to change the structure.
4.5 Exception analysis
4.5.1 Als are required to have in place a process to analyse
exceptions identified through the back-testing of specific risk. This process is intended to serve as the fundamental way in which Als correct their models of specific risk in the event they become inaccurate.

'~ HONG KONG MONETARY AUTHORITY I~., Wm:ffiIll1f:@!m CA·S·5 Use of Internal Models to Measure v. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
4.5.2 There will be a presumption that models that incorporate
specific risk are "unacceptable" if the results at the sub￾portfolio level produce a number of exceptions commensurate with the "red zone" as defined in section
7. Als with "unacceptable" specific risk models are
expected to take immediate action to correct the problem in the model and to ensure that there is a sufficient capital buffer to absorb the risk that the back-test showed had not been adequately captured.
5. Model review
5.1 Acceptability criteria
5.1.1 In reviewing an AI's internal model, the HKMA will
require assurance that:

• the formulae used in the calculation process as
  well as for the pricing of options and other complex instruments are appropriate;
• the structure of internal models is appropriate with
  regard to the investment fund's portfolio;
• the model provides a reliable measure of potential
  losses over time by reviewing the results of the AI's back-tests, Le. comparing value-at-risk measures with actual profits and losses of the fund); and
• data flows and processes associated with the
  model are transparent and accessible. In particular, it is essential that the HKMA should be given easy access, whenever it considers it necessary, to the models' specifications and parameters.
  5.2 Portfolio testing
  5.2.1 From time to time the HKMA may require Als to
  participate in a portfolio-testing exercise. Such an exercise calls for Als using the model to calculate the capital charge on test portfolios determined by the HKMA.

., ~m:&lIl~l!I!rni
CA-S-5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
5.2.2 The purpose of the exercise is to ensure consistency
across all Als.
5.2.3 The results of such an exercise may be taken into
account in determining the multiplication factor applied to the results of an internal model (see section 3.4 above).
5.3 Recognition of models
5.3.1 The model review process will entail an on-site visit
before a model is recognised as acceptable for calculating the market risk capital charge for guaranteed investment funds.
5.3.2 After the initial recognition of models by the HKMA, Als
must inform the HKMA of any subsequent change to the model. The HKMA will determine whether the new methodology is acceptable for calculating the market risk capital charge.
6. The use of back-testing
6.1 Introduction
6.1.1 This section presents the framework for incorporating
back-testing into the internal models approach to market risk capital requirements. It expands on section 3.3.
6.1.2 The process whereby daily profits and losses are
compared with model generated risk measures to gauge the quality and accuracy of a risk measurement system is known as back-testing. As a technique for evaluating the quality of an AI's risk measurement model, back￾testing continues to evolve. New approaches to back￾testing are still being developed and discussed within the risk management community. At present, different Als pertorm different types of back-testing comparisons and the standards of interpretation also differ somewhat across Als.
6.1.3 The essence of all back-testing is the comparison of
actual trading results with model generated risk measures. If this comparison reveals limited differences, the back-test raises no issue regarding the quality of the risk measurement model. In some cases, however, the

• :m=~~lI!fjfl!l!r.u
  CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes comparison may uncover a great number of or significant differences that indicate that problems must almost certainly exist either with the model or with the assumptions of the back-test. Between these two cases is a grey area where the test results are, on their own, inconclusive.
  6.1.4 In considering how to incorporate back-testing more
  closely into the internal models approach to market risk capital requirements, the HKMA seeks to reflect both the fact that the industry has not yet settled on a single back￾testing methodology and that there are concerns over the imperfect nature of the signal generated by back￾testing.
  6.1.5 The remainder of this section describes the back-testing
  framework that is to accompany the internal models capital requirement. The next subsection deals with the nature of the back-tests themselves whilst the subsection that follows relates to the supervisory Interpretation of the results and sets out the standards of the HKMA in this regard.
  6.2 Description of the back-testing framework
  6.2.1 Back-testing programmes consist of a periodic
  comparison of the AI's daily value-at-risk measures with the subsequent daily profit or loss ("trading outcome"). The value-at-risk measures are intended to be larger than all but a certain fraction of the trading outcomes, where that fraction is determined by the confidence level of the value-at-risk measure. Comparing the risk measures with the trading outcomes means that the AI counts the number of times the risk measures were larger than the trading outcome. The fraction actually covered can then be compared with the intended level of coverage to gauge the performance of the AI's risk model.
  6.2.2 The back-tests to be applied for capital adequacy
  purposes will compare whether the observed percentage of outcomes covered by the risk measure is consistent with a 99% level of confidence. That is, they attempt to

I@t) HONG KONG MONETARY AUTHORITY ., 1!f~~lIItr$rni CA·S·5 Use of Internal Models to Measure V.1 ·16.02.01 Market Risks for Investment Guarantees under MPF Schemes determine if an AI's 99th percentile risk measures truly cover 99% of the investment fund's trading outcomes.
6.2.3 The back-testing framework requires the comparison of
daily trading outcomes with a value-at-risk measure based on a one-day holding period. This differs from the requirement set out in subsection 3.3. This requirement reduces the contamination arising from changes in portfolio composition during the holding period which is reflected in actual profit and loss outcomes but not in value-at-risk measures which assume a static portfolio.
6.2.4 The same concerns about contamination of the trading
outcomes continue to be relevant, however, even for one-day trading outcomes. A more sophisticated approach would involve a detailed attribution of profit and loss outcomes by source, including fees, spreads, market movements and intra-day trading results. In such a case the value-at-risk measures can then be compared with the outcomes arising from market movements alone.
6.2.5 To the extent that the back-testing programme is viewed
purely as a statistical test of the integrity of the calculation of the value-at-risk measure, it is more appropriate to employ a definition of daily trading outcome that allows for an uncontaminated test. To meet this standard, Als should develop the capability to perform back-tests based on the hypothetical changes in portfolio value that would occur, were end-of-day positions to remain unchanged during the holding period.
6.2.6 The HKMA encourages Als to develop the capability to
perform back-tests using both hypothetical and actual trading outcomes. Each approach has its own value. In combination, the two approaches are likely to provide a strong understanding of the relation between calculated risk measures and trading outcomes.
6.2.7 The back-testing framework to be applied entails a
formal testing and counting of exceptions on a quarterly basis using the most recent twelve months of data. The framework adopted by the HKMA is to count the number of times that the trading outcomes are not covered by the risk measures ("exceptions"). For example, over 200

CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes trading days, a 99% daily risk measure should cover, on average, 198 of the 200 trading outcomes, leaving two exceptions. Using the most recent twelve months of data yields approximately 250 daily observations for the purposes of back-testing. The HKMA will use the number of exceptions (out of a sample of 250) generated by the AI's model as the basis for a supervisory response, which in serious cases means that the HKMA will need to impose an increase in an AI's capital requirement or disallow use of the model.
6.2.8 The formal implementation of the back-testing
programme should begin on the date that the internal models for measuring market risks of guaranteed investment funds became effective. This implies that the first formal accounting of exceptions under the back￾testing programme would occur a year later. This does not preclude the HKMA from requesting baCk-testing results prior to that date and in particular does not preclude their usage as part of the internal model approval process.
7. The interpretation of back-testing results
7.1 Three zone approach
7.1.1 With the statistical limitations of back-testing in mind the
HKMA has established a framework for the supervisory interpretation of back-testing results that encompasses a range of possible responses, depending on the strength of the signal generated from the back-test. These responses are classified into three zones, distinguished by colours into a hierarchy of responses.
7.1.2 The green zone corresponds to back-testing results that
do not suggest a problem with the quality or accuracy of an AI's model.
7.1.3 The yellow zone encompasses results that raise
questions in this regard but where such a conclusion is not definitive. The back-testing results could be consistent with either accurate or inaccurate models. In such cases the HKMA will require the AI to present additional information about its models.

@i) HONG KONG MONETARY AUTHORITY \. 1fm~iIl'ifl!l.!Ji'U CA·S·5 Use of Internal Models to Measure v. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
7.1.4 The red zone indicates a back-testing result that almost
certainly indicates a problem with an AI's risk model and the HKMA will require some actions to be initiated.
7.1.5 The table in subsection 7.5 sets out the boundaries for
these zones and the presumptive supervisory response for each back-testing outcome, based on a sample of 250 observations. Where back-testing indicates weaknesses in an AI's model the minimum multiplication factor (see section 3.4) of three will be increased. This increment is referred to below as the "plus factor".
7.2 Green zone
7.2.1 This comprises the range of zero to four exceptions.
7.2.2 Since a model that truly provides 99% coverage would
be quite likely to produce as many as four exceptions in a sample of 250 outcomes, there is little reason for concern raised by back-testing results that fall in this range.
7.2.3 In such a case, the minimum multiplication factor of three
will be applied to the value-at-risk outcome, provided that the AI has already met all the qualitative criteria.
7.3 Yellow zone
7.3.1 The range from five to nine exceptions constitutes the
yellow zone.
7.3.2 Outcomes in this range are plausible for both accurate
and inaccurate models, although they are generally more likely for inaccurate models than for accurate models. Moreover, the presumption that the model is inaccurate should grow as the number of exceptions increases in the range from five to nine.
7.3.3 Within the yellow zone, the number of exceptions should
generally guide the size of potential supervisory increases in an AI's capital requirement for guaranteed investment funds.
7.3.4 The table in subsection 7.5 sets out the guidelines for
increases in the multiplication factor applicable to the internal models capital requirement resulting from back￾testing results falling in the yellow zone.

i@tJ HONG KONG MONETARY AUTHORITY

• 1!tm~llltrJM.F.U
  CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
  7.3.5 It should be stressed, however, that these increases are
  not meant to be purely automatic. Results in the yellow zone do not always imply an inaccurate model and the HKMA has no interest in penalising Als solely for bad luck. Nevertheless, to keep the incentives aligned properly, back-testing results in the yellow zone should generally be presumed to apply an increase in the multiplication factor unless the AI can demonstrate that such an increase is not warranted. This means the burden of proof that the model is fundamentally sound rests with the AI concerned.
  7.3.6 In such a situation, there are many different types of
  additional information that might be relevant to an assessment of the AI's model. For example, it would then be particularly valuable to see the results of back￾tests covering disaggregated sub-sets of the positions. Als that engage in regular back-testing programmes may break up their investment fund's portfolio into units organised around risk factors or product categories. Disaggregating in this way could allow the tracking of a problem that surfaced at the aggregate level back to its source at the level of a specific trading unit or risk model.
  7.3.7 Als should document all of the exceptions generated
  from their continuing back-testing programme, including an explanation for the exceptions. This documentation is important for determining an appropriate supervisory response to back-testing results falling in the yellow zone. Als may also implement back-testing for confidence intervals other than the 99th percentile or may perform other statistical tests not considered here. This information could also prove very helpful in assessing their model.
  7.3.8 In practice, there are several possible explanations for a
  back-testing exception:
• the basic integrity of the model;
• an under-specified or low quality model,
• poor intra-day trading results; or
• pure bad luck.

.... if~~lJi!l~l!I!mi
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes Classifying the exceptions generated by an AI's model into these categories can be a very useful exercise, as discussed in the next section.
7.4 Red zone
7.4.1 The red zone encompasses ten or more exceptions.
7.4.2 In contrast to the yellow zone, where the HKMA may
exercise judgement in interpreting the back-testing results, outcomes in the red zone should generally lead to an automatic presumption that a problem exists with an AI's model. It is extremely unlikely that an accurate model would independently generate ten or more exceptions from a sample of 250 trading outcomes.
7.4.3 In general, therefore, if an AI's model falls into the red
zone, the HKMA will automatically increase the multiplication factor applicable to its model by from three to four.
7.4.4 The HKMA will also investigate why the AI's model
produced such a large number of exceptions and require the AI to begin work on improving its model immediately. In the case of a severe problem with the basic integrity of the model, the HKMA may disallow the use of the model for capital purposes.
7.4.5 Although ten exceptions is a very high number for 250
observations, there will on very rare occasions be a valid reason for an accurate model to produce so many exceptions. In particular, when financial markets are subject to a major regime shift, many volatilities and correlations can be expected to shift as well, perhaps substantially. Unless an AI is prepared to update its volatility and correlation estimates instantaneously, such a regime shift could generate a number of exceptions in a short period of time. In essence, however, these exceptions would all be occurring for the same reason and therefore the appropriate supervisory reaction might not be the same as when there were ten exceptions but each from a separate incident. One possible supervisory response in this instance would be simply to require the AI's model to take account of the regime shift as quickly

HONG KONG MONETARY AUTHORITY
., i!fm~I!l~~r.u
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes as it can while maintaining the integrity of its procedures for updating the model.
7.4.6 It should be stressed, however, that this exception
should be allowed only under the most extraordinary circumstances. The HKMA is committed to an automatic and non-discretionary increase in an AI's capital requirement for back-testing results that fall into the red zone.
7.5 Plus factor for back-testing exceptions
Number of
Zone exceptions Plus Factor
Green zone a 0.00
1 0.00
2 0.00
3 0.00
4 0.00
Yellow zone 5 0.40
6 0.50
7 0.65
8 0.75
9 0.85
Red zone 10 or more 1.00
8. Classification of exceptions
8.1 Reasons for exceptions
8.1 1 Exceptions will normally fall into the following categories:

• the basic integrity of the model is flawed;
• its accuracy needs improving;
• bad luck or markets have moved in a fashion
  unanticipated by the model; and
• intra-day trading.

HONG KONG MONETARY AUTHORITY

• =m:m:&Jill'1flM.F.U
  CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes
  8.1.2 Samples of exceptions arising from flawed integrity of the
  model could be that:
• the AI's systems are not capturing the risk of the
  investment fund's positions, e.g. the positions of the investment fund are being reported incorrectly; or
• model volatilities or correlations are calculated
  incorrectly, e.g. the computer is dividing by 250 when it should be dividing by 225.
  8.1.3 A sample inaccuracy in a model could be that:
• the risk measurement model is not assessing the
  risk of some instruments with sufficient precision, e.g. too few maturity buckets or a spread omitted.
  8.1.4 Sample reasons for exceptions resulting from bad luck or
  unanticipated market movements could be:
• random chance (a very low probability event);
• markets moved by more than the model predicted
  was likely, i.e. volatility was significantly higher than expected; or
• markets did not move together as expected, i.e.,
  correlations were significantly different than what was assumed by the model.
  8.1.5 Intra-day trading can cause exceptions, e.g.
• there was a large (and money losing) change in the
  positions or some other income events between the end of the first day, when the risk estimate was calculated, and the end of the second day, when trading results were tabulated.
  8.2 Discussion of categories
  8.2.1 In general, problems relating to the basic integrity of the
  risk measurement model are potentially the most serious. If exceptions are attributable to this category for a particular trading unit, the plus factor should apply. In addition, the model may be in need of a substantial review or adjustment. The HKMA may disallow the use of

., ifl=~~I!l'lf@fiU
CA·S·5 Use of Internal Models to Measure V. 1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes internal models altogether until appropriate corrections are undertaken by the AI.
8.2.2 The second category of problem, lack of model precision,
can be expected to occur at least part of the time with most risk measurement models. No model can hope to achieve infinite precision as all models involve some amount of approximation. If, however, a particular AI's model appears more prone to this type of problem than others, the HKMA will impose the plus factor and will also consider other incentives to encourage improvement.
8.2.3 The third category of problem, unanticipated market
movements, should also be expected to occur at least some of the time with value-at-risk models. Even an accurate model cannot be expected to cover 100% of trading outcomes. Some exceptions will be the random 1% that the model can be expected not to cover. In other cases, the behaviour of the markets may shift so that previous estimates of volatility and correlation are less appropriate. No value-at-risk model will be immune from this type of problem; it is inherent in the reliance on past market behaviour as a means of gauging the risk of future market movements. Exceptions for such reasons do not suggest a problem. If, however, the shifts in volatilities and correlations are deemed to be permanent, the HKMA may require the AI to recalculate its value at risk using volatilities and correlations based on a shorter observation period.
8.2.4 Depending on the definition of trading outcomes
employed for the purpose of back-testing, exceptions could also be generated by intra-day trading results or an unusual event in trading income other than from positioning. Although exceptions arising from these reasons may not necessarily suggest a problem with the value-at-risk model, they could still be causes for concern. The imposition of the plus factor would be considered.
8.2.5 Another consideration is the extent to which a trading
outcome exceeds the risk measure. With all other things being equal, exceptions generated by trading outcomes far in excess of the risk measure are of greater concern

• 1!=rt!hiElIl~@Jm
  CA·S·5 Use of Internal Models to Measure V.1 - 16.02.01 Market Risks for Investment Guarantees under MPF Schemes than those outcomes which are only slightly larger than the risk measure.
  8.3 Consideration of factors by HKMA
  8.3.1 In deciding whether to apply increases in an AI's capital
  requirement, the HKMA will weigh these factors in addition to an appraisal of the AI's compliance with applicable qualitative standards of risk management. Based on the additional information provided by the AI, the HKMA will decide on the appropriate course of action.
  8.3.2 In general, the imposition of a higher capital requirement
  for outcomes in the yellow zone is an appropriate response if the HKMA believes the reason for being in the yellow zone is a problem in an AI's model which can be corrected. This can be contrasted with the case of an unexpected bout of high market volatility, which nearly all models may fail to predict. While these episodes may be stressful, they do not necessarily indicate that an AI's risk model is in need of redesign.
  8.3.3 In the case of severe problems with the basic integrity of
  the model, the HKMA will consider whether to disallow the use of the model for capital purposes altogether. Contents Glossary Home Introduction